Up-conversion of video streams uses frame interpolation to insert intermediate frames between original frames of a video stream. For example, up-converting a video stream from 24 frames per second (fps), as in conventional analog video, to 120 fps, as in high-definition television, is performed by inserting four intermediate frames between consecutive or temporally adjacent frames of the original video stream.
Intermediate frames can be interpolated based on analyzed motions of pixels between two adjacent frames of an original video stream. However, certain situations make such interpolation difficult. In particular, movement of a foreground object relative to a background image may result in image pixels that are visible in only one of the two adjacent video frames. This may be caused by factors such as reveal and occlusion of background regions, as well as by object deformations.
The embodiments described below conceal errors at a per-pixel level. The pooling of per-pixel errors is limited to a small region in the neighborhood and therefore may be called “local error concealment” strategies. These strategies, together with other strategies that may be applied on a more global basis, help reduce visible artifacts in motion compensated up-conversion systems.
Errors in motion based up-conversion occur on account of inaccurate handling of reveal and occluded regions, object deformations, situations where object motion is larger than the motion search range, etc. Of specific interest are situations where the region of error is limited to a small area of the screen. These errors can be hidden with robust local error concealment strategies as described below.
The background description provided herein is for the purpose of generally presenting the context of the disclosure. Work of the presently named inventors, to the extent it is described in this background section, as well as aspects of the description that may not otherwise qualify as prior art at the time of filing, are neither expressly nor impliedly admitted as prior art against the present disclosure.